Work assistance system, work assistance method, and computer-readable recording medium

ABSTRACT

A work assistance system includes: a memory; and a processor coupled to the memory, wherein the processor executes a process including: displaying a data list in which pieces of data are arranged according to a predetermined rule, in a state being viewable by an operator; detecting a direction of a gaze movement of the operator with respect to the displayed data list; and changing a region to be highlighted in the data list, depending on whether the detected direction of the gaze movement is in a predetermined state.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of International Application No.PCT/JP2015/079473, filed on Oct. 19, 2015, the entire contents of whichare incorporated herein by reference.

FIELD

The embodiments discussed herein are related to a work assistancesystem, a work assistance method, and a computer-readable recordingmedium.

BACKGROUND

In government offices, banking facilities, and the like, for example,there is a case where a code number related to an application content ora code number for identifying a banking facility to be transferred intois entered in or input to a form. A user who enters the code numbers inthe form refers to, for example, a code list and finds a code numbercorresponding to the entry content, to identify the code number.

Patent Literature 1: Japanese Laid-open Patent Publication No.2005-309633

Patent Literature 2: Japanese Laid-open Patent Publication No.2013-37399

However, even if the user is aware that he needs to enter a code numberin the form, the user may feel confused as to how to find out the codenumber. At this time, for example, even if an example of descriptions inthe form is displayed as a reference image and a reference image area isdisplayed for a content to be input to an entry column, it is difficultto assist the user to find out a desired code number from the code list.

In one aspect, an object of the present invention is to provide a workassistance system, a work assistance method, and a computer-readablerecording medium that can assist a work of identifying desired data froma data list.

SUMMARY

According to an aspect of an embodiment, a work assistance systemincludes: a memory; and a processor coupled to the memory, wherein theprocessor executes a process including: displaying a data list in whichpieces of data are arranged according to a predetermined rule, in astate being viewable by an operator; detecting a direction of a gazemovement of the operator with respect to the displayed data list; andchanging a region to be highlighted in the data list, depending onwhether the detected direction of the gaze movement is in apredetermined state.

The object and advantages of the invention will be realized and attainedby means of the elements and combinations particularly pointed out inthe claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and arenot restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an example of a configuration ofa work assistance system according to a first embodiment;

FIG. 2 is a diagram illustrating an example of a list storage unit;

FIG. 3 is a diagram illustrating an example when referring to a datalist at the time of filling in a form;

FIG. 4 is a diagram illustrating another example when referring to adata list at the time of filling in a form;

FIG. 5 is a flowchart illustrating an example of work assistanceprocessing according to the first embodiment;

FIG. 6 is a block diagram illustrating an example of a configuration ofa work assistance system according to a second embodiment;

FIG. 7 is a diagram illustrating an example of a guide display and adirection of a gaze movement;

FIG. 8 is a flowchart illustrating an example of work assistanceprocessing according to the second embodiment; and

FIG. 9 is a diagram illustrating an example of a computer that executesa computer-readable recording medium.

DESCRIPTION OF EMBODIMENTS

Preferred embodiments of the present invention will be explained withreference to accompanying drawings. Disclosed techniques are not limitedto the embodiments. Embodiments described below can be combined with oneanother as appropriate within a range of not causing any contradiction.

[a] First Embodiment

First, a case where a user has difficulty in filling in a form or thelike is described. A first case is, for example, a case where it isdifficult for a user to decide whether to enter personal informationwhen he fills in a street survey, that is, the user feels awkwardbecause he does not wish to answer. A second case is, for example, acase where a user has visited a store to send a parcel with a memonoting an address and a name, but has found that a phone number alsoneeds to be filled in and then tries to remember the phone number, thatis, the user feels awkward because he does not remember the phonenumber.

A third case is, for example, a case where the content to be writtensuch as user's name is clear, but a user feels awkward because he doesnot know where to enter his name in the form. A fourth case is, forexample, a case where a user does not know how to look up a code list atthe time of entering a prefecture name by converting it to a codenumber, that is, the user feels awkward because he does not know how tofill in the form. In the fourth case, when the list is two or moredimensional, the user takes time until understanding how to look up thelist, and thus it is effective to display a reference method to the listwithout any confusion about how to read the list as a guide.

FIG. 1 is a block diagram illustrating an example of a configuration ofa work assistance system according to a first embodiment. A workassistance system 1 illustrated in FIG. 1 is a system that assists auser to refer to a list without feeling confused, with respect to thefourth case described above. That is, the work assistance system 1 is asystem that assists a user to find a desired code number from a codelist, for example, at the time of filling a code number in a form.

The work assistance system 1 illustrated in FIG. 1 includes a displayunit 10 that displays a data list, which is a list of data to bereferred to, for example, when a user, that is, an operator fills in aform on a writing table, a gaze sensor 11 for detecting a gaze of theuser, and an information processing device 100. The informationprocessing device 100 displays the data list in which pieces of data arearranged according to a predetermined rule, on the display unit 10 in astate being viewable by the operator. The information processing device100 detects a direction of a gaze movement of the operator with respectto the data list displayed on the display unit 10, based on informationobtained by the gaze sensor 11. The information processing device 100changes a region to be highlighted in the data list, depending onwhether the detected direction of the gaze movement is in apredetermined state. Accordingly, the work assistance system 1 canassist the work of identifying desired data from the data list.

The display unit 10 is a display device to display various pieces ofinformation. The display unit 10 is realized, for example, by aprojector or a liquid-crystal display as a display device. The displayunit 10 displays various screens such as a reference screen input fromthe information processing device 100. The display unit 10 is, forexample, a projector arranged at a position where the reference screencan be projected to a part of the writing table, or a liquid-crystaldisplay incorporated in the writing table.

The gaze sensor 11 is a sensor that detects the gaze of a user who fillsin a form on the writing table. The gaze sensor 11 is realized by, forexample, an infrared ray camera. The gaze sensor 11 captures an image ofpupil-cornea reflection light (Purkinje image) of the user, and outputsthe captured image to the information processing device 100.

A configuration of the information processing device 100 is describednext. As illustrated in FIG. 1, the information processing device 100includes a storage unit 120 and a control unit 130. The informationprocessing device 100 can include known various types of functionalunits included in a computer, for example, functional units such asvarious types of input devices and voice output devices, other than thefunctional units illustrated in FIG. 1.

The storage unit 120 is realized, for example, by a semiconductor memorydevice such as a RAM (Random Access Memory) or a flash memory, or astorage device such as a hard disk or an optical disk. The storage unit120 includes a list storage unit 121. The storage unit 120 also storestherein information to be used for processing performed by the controlunit 130.

The list storage unit 121 stores therein a list to be referred to. Thelist is a data list in which pieces of data are arranged according to apredetermined rule. FIG. 2 is a diagram illustrating an example of thelist storage unit. The list illustrated in FIG. 2 is, for example, alist of recycling fees of recycling tickets of home electricalappliances. As illustrated in FIG. 2, the list storage unit 121 includesitems of “manufacturer name (code)” and “product (code)”.

The “manufacturer name (code)” is information indicating a manufacturername of home electrical appliances and a code thereof. Further, the“manufacturer name (code)” corresponds to, for example, first key datain the data list. The “product (code)” is information indicating aproduct name of home electrical appliances to be recycled and a codethereof. The “product (code)” further includes items of “air conditioner(10)”, “television”, and “refrigerator (30)”. The “television” furtherincludes items of “CRT-based (20)” and “LCD/plasma based (50)”. The“product (code)” corresponds to, for example, second key data in thedata list. The first key data and the second key data can be set in atwo-dimensional list, for example, such that data of one axis of a largelist, which is desired to be referred to over a plurality of pages, isset as the first key data, and data of the other axis is set as thesecond key data. In the example of FIG. 2, it is indicated that, forexample, the recycling fee of a product “refrigerator (30)” having amanufacturer name “AAA (001)” is “3,672 Yen”.

Returning to the description of FIG. 1, the control unit 130 isrealized, for example, by a CPU (Central Processing Unit) or an MPU(Micro Processing Unit) that executes a program stored in a storagedevice therein, using a RAM as a work area. Further, the control unit130 can be realized by an integrated circuit, for example, an ASIC(Application Specific Integrated Circuit) or an FPGA (Field ProgrammableGate Array). The control unit 130 includes a display control unit 131, adetection unit 132, and a highlighting unit 133, and realizes orperforms functions and actions of information processing describedbelow. The internal configuration of the control unit 130 is not limitedto the configuration illustrated in FIG. 1, and other types ofconfigurations can be employed as far as these are configurations inwhich information processing described later is performed.

When having detected that a form is placed on a writing table, forexample, by a sensor (not illustrated), the display control unit 131refers to the list storage unit 121 to display a reference screenincluding the data list on the display unit 10, and projects thereference screen on the writing table. That is, the display control unit131 displays the data list in a state being viewable by a user, who isan operator.

The display control unit 131 displays a reference region guide on thedata list of the reference screen. The reference region guide is adisplay or the like in which a manufacturer name on a vertical axis ofthe data list is enclosed by a frame. Further, when highlightinginformation is input from the highlighting unit 133, the display controlunit 131 highlights the guide region according to the highlightinginformation. After having displayed the data list and the referenceregion guide, the display control unit 131 outputs guide regioninformation and a detection start instruction to the detection unit 132.The display control unit 131 can output the guide region information andthe detection start instruction to the detection unit 132 withoutdisplaying the reference region guide. In this case, the display of thereference region guide can be highlighted.

When the guide region information and the detection start instructionhave been input from the display control unit 131, the detection unit132 detects a gaze of a user based on a captured image input from thegaze sensor 11. That is, the detection unit 132 detects a direction of agaze movement of the user with respect to the displayed list.

The detection unit 132 determines whether the direction of the gazemovement is along a direction of searching the guide region, in otherwords, a direction of the gaze movement different from the direction ofsearching the guide region has been detected, based on the guide regioninformation, and the detected direction of the gaze movement. If adirection of the gaze movement different from the direction of searchingthe guide region has been detected, the detection unit 132 outputs theguide region information and a highlighting instruction to thehighlighting unit 133. If the direction of the gaze movement differentfrom the direction of searching the guide region has not been detected,the detection unit 132 continues detection of the user's gaze.

Upon input of the guide region information and the highlightinginstruction from the detection unit 132, the highlighting unit 133generates highlighting information indicating that the guide region inthe guide region information is to be highlighted, and outputs thegenerated highlighting information to the display control unit 131. Thatis, the highlighting unit 133 changes a region to be highlighted in thedata list depending on whether the direction of the gaze movementdetected by the detection unit 132 is in a predetermined state. Thehighlighting is at least one display among a display in which luminanceis increased, a display in which luminance is periodically changed, adisplay in which color is changed, and a display in which a thickness ofa frame enclosing a region is changed, for example, as compared withother regions in the data list.

That is, if the direction of the gaze movement is in a first state, thehighlighting unit 133 does not perform highlighting, and if thedirection of the gaze movement is in a different state from the firststate, the highlighting unit 133 highlights a specific region in thedata list. The first state is a predetermined state and is a state wherea direction of the gaze movement different from the direction ofsearching the guide region is not detected. That is, the first state isa state decided according to a predetermined rule followed by the datalist. In other words, the first state indicates, for example, a casewhere the user's gaze position is moving downward in a region of thefirst key data in the data list. The predetermined rule followed by thedata list is, for example, a rule in which the first key data isarranged on a vertical axis and the second key data is arranged on ahorizontal axis.

If the display control unit 131 does not display the reference regionguide, the highlighting unit 133 can display the reference region guidewhen the direction of the gaze movement is different from that of thefirst state. Further, even if the display control unit 131 displays thereference region guide, if the direction of the gaze movement isdifferent from that of the first state, the highlighting unit 133 canoutput a new guide so that the reference region can be referred to bythe user ahead of the gaze. The new guide is, for example, at least onedisplay among a display in which a pop-up text is displayed, a displayin which a text is scrolled so as to cross the gaze, and a display inwhich an arrow icon indicating a direction of leading the user's gaze toa predetermined gaze position is output.

A specific example of the first embodiment is described with referenceto FIG. 3 and FIG. 4. FIG. 3 is a diagram illustrating an example whenreferring to a data list at the time of filling in a form. Asillustrated in FIG. 3, when a user 20 places a form 22 on a writingtable 21, a reference screen 24 is displayed in a reference screenregion 23 on the writing table 21. The gaze sensor 11 is provided on thewriting table 21 to detect a gaze of the user 20. On the referencescreen 24, a reference region guide 25 is displayed in a region of thefirst key data on the vertical axis indicating the manufacturer name, ofthe data list on the reference screen 24. Further, in a reference screenregion 23, a guide 26 by characters, for example, “please find themanufacturer name first” is displayed. Also, in the reference screenregion 23, a previous page button 27 and a next page button 28 forshifting the page are displayed when the data list is displayed over theplurality of reference screens 24. In the state illustrated in FIG. 3,the gaze of the user 20 moves sequentially from the top to the bottom inthe region of the first key data, as indicated by an arrow 29. Whenhaving found the manufacturer name being the first key data, the user 20reads a code of the found manufacturer name and enters the code in anentry column 30 of the form.

FIG. 4 is a diagram illustrating another example when referring to adata list at the time of filling in a form. In the state illustrated inFIG. 4, the gaze of the user 20 is outside the region of the first keydata indicated by the reference region guide 25 and is located near thesecond key data of the horizontal axis in the data list. That is, theuser 20 feels awkward because of not knowing where to refer to. In thiscase, the information processing device 100 blinks the region of thefirst key data on the vertical axis indicating, for example, themanufacturer name, the reference region guide 25, and the guide 26 toperform highlighting, thereby assisting the user 20. Alternatively, theinformation processing device 100 displays an arrow icon for leading thegaze in the direction of the reference region guide 25, to assist theuser 20. Thus, the work assistance system 1 can assist the user to knowwhere to refer to in the data list.

Further, in FIG. 3 and FIG. 4, when the gaze of the user 20 is in theregion of the first key data, the reference region guide with respect toa cell near the gaze position is highlighted, and when the gaze of theuser 20 is not in the region of the first key data, the reference regionguide 25 including the entire region of the first key data beingdisplayed can be highlighted. That is, the work assistance system 1changes a region to be highlighted in the data list.

Next, an operation of the work assistance system 1 according to thefirst embodiment is described. FIG. 5 is a flowchart illustrating anexample of work assistance processing according to the first embodiment.

When having detected that a form is placed on a writing table, forexample, by a sensor (not illustrated), the display control unit 131 ofthe information processing device 100 displays a reference screenincluding a data list on the display unit 10 (writing table) (Step S1).The display control unit 131 displays a reference region guide on thedata list on the reference screen (Step S2). After having displayed thedata list and the reference region guide, the display control unit 131outputs guide region information and a detection start instruction tothe detection unit 132.

When the guide region information and the detection start instructionhave been input from the display control unit 131, the detection unit132 detects a gaze of a user based on a captured image input from thegaze sensor 11. The detection unit 132 determines whether a direction ofthe gaze movement different from a direction of searching the guideregion has been detected, based on the guide region information, and thedetected gaze position and movement (Step S3). If the direction of thegaze movement different from the direction of searching the guide regionhas been detected (YES at Step S3), the detection unit 132 outputs theguide region information and a highlighting instruction to thehighlighting unit 133.

Upon input of the guide region information and the highlightinginstruction from the detection unit 132, the highlighting unit 133generates highlighting information indicating that the guide region inthe guide region information is to be highlighted, and outputs thegenerated highlighting information to the display control unit 131. Uponinput of the highlighting information from the highlighting unit 133,the display control unit 131 highlights the guide region according tothe highlighting information (Step S4), to finish the processing whenthe user finishes referring to the data. If the direction of the gazemovement different from the direction of searching the guide region hasnot been detected (NO at Step S3), the detection unit 132 continuesdetection of the user's gaze and finishes the processing when the userfinishes referring to the data. Accordingly, the work assistance system1 can assist the work of identifying the desired data from the datalist.

In this manner, the work assistance system 1 displays the data list inwhich pieces of data are arranged according to the predetermined rule ina state being viewable by an operator. Further, the work assistancesystem 1 detects a direction of a gaze movement of the operator withrespect to the displayed data list. Further, the work assistance system1 changes the region to be highlighted in the data list, depending onwhether the detected direction is in the predetermined state. As aresult, the work assistance system 1 can assist the work of identifyingthe desired data from the data list.

Further, when the direction of the gaze movement is in the first state,the work assistance system 1 does not perform highlighting, and when thedirection of the gaze movement is in a state different from the firststate, the work assistance system 1 highlights a specific region in thedata list. As a result, only when the user's gaze is directed to aregion other than the specific region to be referred to, the workassistance system 1 can remind the user of the specific region to bereferred to.

Further, in the work assistance system 1, the first state is a statedecided according to the predetermined rule followed by the data list.As a result, the work of identifying the desired data according to thedata list can be assisted.

Further, in the work assistance system 1, the highlighting is at leastone display among a display in which luminance is increased, a displayin which luminance is periodically changed, a display in which color ischanged, and a display in which a thickness of a frame enclosing aregion is changed as compared with other regions in the data list.Further, the work assistance system 1 can output a new guide so that thereference region can be referred to by the user ahead of the user'sgaze. The new guide is, for example, at least one display among adisplay in which a pop-up text is displayed, a display in which a textis scrolled so as to cross the gaze, and a display in which an arrowicon indicating a direction of leading the user's gaze to apredetermined gaze position is output. As a result, the user can findthe specific region to be referred to.

Further, in the work assistance system 1, the operator's condition canbe presented on other information processing devices and otherinformation presentation screens located on a place where the system isinstalled (for example, a store), in synchronization with thehighlighting to a state viewable by the operator. As a result, forexample, a dedicated assistant can directly assist the user with thework.

[b] Second Embodiment

In the first embodiment described above, a region in the data list ishighlighted according to the user's gaze. However, a specific operationwith respect to any data in the data list can be received. An embodimentin this case is described as a second embodiment. FIG. 6 is a blockdiagram illustrating an example of a configuration of a work assistancesystem according to the second embodiment. Constituent elementsidentical to those of the work assistance system 1 according to thefirst embodiment are denoted by like reference signs and redundantexplanations of the constituent elements and operations are omitted.

A work assistance system 2 according to the second embodiment includes apen input sensor 12 in addition to the units provided in the workassistance system 1 according to the first embodiment, to receive user'spen input operation.

The pen input sensor 12 is a sensor that detects a pen input by a userwho fills in a form on a writing table. The pen input sensor 12 isrealized, for example, by a digitizer installed on the writing table. Aninput range of the pen input sensor 12 is set, for example, to a rangeincluding a display range (a projection range) on a reference screendisplayed by the display unit 10 on the writing table and a range inwhich entry to the form is performed. The pen input sensor 12 detects apen point position to generate operation information based on the penpoint position, and outputs the generated operation information to aninformation processing device 200. That is, the pen input sensor 12detects the pen point position on the reference screen or on the form,generates the operation information based on the detected pen pointposition, and outputs the operation information to the informationprocessing device 200.

The information processing device 200 of the work assistance system 2includes a display control unit 231, a detection unit 232, and ahighlighting unit 233 instead of the display control unit 131, thedetection unit 132, and the highlighting unit 133 of the informationprocessing device 100 according to the first embodiment. The informationprocessing device 200 further includes a reception unit 234 and acalculation unit 235. In the following descriptions, features differentfrom those of corresponding constituent elements according to the firstembodiment are mainly explained and descriptions of identical functionsare omitted.

When having detected that a form is placed on a writing table, forexample, by a sensor (not illustrated), the display control unit 231displays a message for prompting entry to a name or address entry columnprior to the entry of a code number, on the display unit 10. The displaycontrol unit 231 detects that the form is placed on the writing table bycapturing an image of a QR (Quick Response) code, for example, providedin the form by a camera (not illustrated). Further, the display controlunit 231 outputs a gaze calibration instruction to the detection unit232 and the reception unit 234.

When gaze calibration completion information is input from the detectionunit 232, the display control unit 231 refers to the list storage unit121 to display a reference screen including a data list on the displayunit 10, and projects the reference screen on the writing table.

Similarly to the display control unit 131 according to the firstembodiment, the display control unit 231 displays a reference regionguide on the data list on the reference screen. The display control unit231 displays the reference region guide, for example, on the first axisof the data list, that is, on a vertical axis. After having displayedthe data list and the reference region guide, the display control unit231 outputs guide region information and a detection start instructionto the detection unit 232. Further, when highlighting information orspecific display information is input from the highlighting unit 233,the display control unit 231 highlights a guide region or specific data,according to the highlighting information or the specific displayinformation. Further, when a change instruction is input from thedetection unit 232, the display control unit 231 changes display of thereference region guide based on the change instruction.

When the gaze calibration instruction is input from the display controlunit 231, the detection unit 232 outputs a detected gaze position to thecalculation unit 235. The detection unit 232 corrects user's gazeposition based on a captured image input from the gaze sensor 11 byusing a correction value input from the calculation unit 235. That is,the detection unit 232 corrects the user's gaze position by using a factthat a pen point position of the user who is performing an entryoperation matches the user's gaze position. When the pen point positionof the user departs from the name or address entry column, the detectionunit 232 completes correction of the user's gaze position, that is, gazecalibration. Upon completion of the gaze calibration, the detection unit232 outputs gaze calibration completion information to the displaycontrol unit 231 and the reception unit 234.

When the guide region information and the detection start instructionare input from the display control unit 231, the detection unit 232detects the user's gaze based on the captured image input from the gazesensor 11. The detection unit 232 determines whether a direction of thegaze movement different from a direction of searching the guide regionhas been detected, based on the guide region information, and theposition and the direction of the detected gaze point. If the directionof the gaze movement different from the direction of searching the guideregion has been detected, the detection unit 232 outputs the guideregion information and a highlighting instruction to the highlightingunit 233. If the direction of the gaze movement different from thedirection of searching the guide region has not been detected, thedetection unit 232 continues detection of the user's gaze.

Further, when specific information is input from the reception unit 234,the detection unit 232 changes the content of the first state, which isa determination criterion at the time of changing the region to behighlighted. That is, the detection unit 232 changes, for example, thefirst state corresponding to the region of the first key data on avertical axis of the data list to a first state corresponding to theregion of the second key data on a horizontal axis of the data list,because of the input of the specific information. That is, upon input ofthe specific information from the reception unit 234, the detection unit232 changes the guide region in the guide region information to beoutput to the highlighting unit 233. That is, the guide region ischanged, for example, from the region of the first key data on thevertical axis of the data list to the region of the second key data onthe horizontal axis. Further, after having changed the guide region, thedetection unit 232 outputs a change instruction of the guide region tothe display control unit 231.

When the guide region information and the highlighting instruction areinput from the detection unit 232, the highlighting unit 233 generateshighlighting information indicating that the guide region in the guideregion information is to be highlighted, and outputs the generatedhighlighting information to the display control unit 231. Further, whenthe specific information is input from the reception unit 234, thedetection unit 232 generates specific display information indicatingthat, for example, highlighting is to be performed for the identifiedkey data based on the specific information, and outputs the generatedspecific display information to the display control unit 231.

When the gaze calibration instruction is input from the display controlunit 231, the reception unit 234 outputs operation information includingthe pen point position input from the pen input sensor 12 to thecalculation unit 235. Further, upon input of the gaze calibrationcompletion instruction from the detection unit 232, the reception unit234 starts reception of a specific operation with respect to any onepiece of data in the displayed data list, based on the operationinformation including the pen point position input from the pen inputsensor 12. When having received the specific operation, the receptionunit 234 outputs specific information identifying the data to thedetection unit 232 and the highlighting unit 233.

When the gaze position is input from the detection unit 232 and theoperation information is input from the reception unit 234, thecalculation unit 235 calculates a correction value based on a differencebetween the gaze position and the pen point position in the operationinformation, and outputs the calculated correction value to thedetection unit 232. That is, the calculation unit 235 calculates acorrection value for detecting the gaze position of an individual userwho is performing an entry operation, based on a relation between thepen point position on a screen and the gaze position while the entryoperation to the name or address entry column is being performed.

A specific example of the second embodiment is described with referenceto FIG. 7. FIG. 7 is a diagram illustrating an example of a guidedisplay and a direction of a gaze movement. In FIG. 7, descriptions aremade by using reference screens 31 a to 31 c corresponding to thereference screen 24 on the writing table 21 in FIG. 3. In the followingdescriptions, an example is used in which a code and the fee of aproduct “television” of “CRT-based” under the name of manufacturer “BBD”are searched.

As illustrated in FIG. 7, in a reference screen 31 a, a gaze of a user20 moves sequentially from the top to the bottom in a region 32 of thefirst key data, as indicated by an arrow 33, in order to find themanufacturer name. At this time, a guide 32 a is displayed in the region32 of the first key data. When having found the manufacturer name “BBD”being the first key data, the user 20 reads a code “103” of the foundmanufacturer name and identifies the first key data by touching themanufacturer name “BBD” by a pen point. Further, the user 20 enters theread code “103” of the manufacturer name in a corresponding entry columnof the form. At the time of searching the first key data, if the gaze ofthe user deviates from the region 32 of the first key data, the region32 of the first key data is highlighted.

On the reference screen 31 b, a row corresponding to the identifiedfirst key data 34 is, for example, highlighted. On the reference screen31 b, the gaze of the user 20 moves as indicated by an arrow 36 from theidentified first key data 34 toward a region 35 of the second key datain order to search a product. At this time, a guide 35 a is displayed inthe region 35 of the second key data. The guide 35 a is changed overfrom the guide 32 a and displayed correspondingly to the identificationof the first key data 34 by the user 20. When having found “CRT-based”of the product “television” being the second key data, the user 20 readsa code “20” of the found product, and touches “CRT-based” by the penpoint to identify the second key data. Further, the user 20 enters thecode “20” of the read product in a corresponding entry column of theform. At the time of searching the second key data, if the gaze of theuser deviates from the region 35 of the second key data, the region 35of the second key data is highlighted.

On a reference screen 31 c, by identifying the first key data and thesecond key data on the reference screens 31 a and 31 b, a unique cell inthe data list, that is, a column 38 of the recycling fee is highlighted.At this time, the gaze of the user 20 moves as indicated by an arrow 37.The user 20 reads the amount “1,836 Yen” in the column 38 of therecycling fee and enters the amount in a corresponding entry column ofthe form. Thus, the work assistance system 2 can assist a work ofidentifying desired data from the data list.

Next, the operation of the work assistance system 2 according to thesecond embodiment is described. FIG. 8 is a flowchart illustrating anexample of work assistance processing according to the secondembodiment.

When having detected that a form is placed on a writing table, forexample, by a sensor (not illustrated), the display control unit 231 ofthe information processing device 200 displays a message for promptingentry to a name or address entry column on the display unit 10. Thedisplay control unit 231 also outputs a gaze calibration instruction tothe detection unit 232 and the reception unit 234. Upon input of thegaze calibration instruction from the display control unit 231, thedetection unit 232 outputs a detected gaze position to the calculationunit 235. Further, upon input of the gaze calibration instruction fromthe display control unit 231, the reception unit 234 outputs operationinformation including a pen point position input from the pen inputsensor 12 to the calculation unit 235.

When the gaze position is input from the detection unit 232 and theoperation information is input from the reception unit 234, thecalculation unit 235 calculates a correction value based on a differencebetween the gaze position and the pen point position in the operationinformation (Step S11), and outputs the calculated correction value tothe detection unit 232. The detection unit 232 corrects the user's gazeposition based on a captured image input from the gaze sensor 11 byusing the correction value input from the calculation unit 235. Uponcompletion of the gaze calibration, the detection unit 232 outputs gazecalibration completion information to the display control unit 231 andthe reception unit 234.

Upon input of the gaze calibration completion instruction from thedetection unit 232, the display control unit 231 displays a referencescreen including a data list on the display unit 10 (writing table)(Step S12). The display control unit 231 displays a reference regionguide on the first axis of the data list (Step S13). After displayingthe data list and the reference region guide, the display control unit231 outputs guide region information and a detection start instructionto the detection unit 232.

Upon input of the guide region information and the detection startinstruction from the display control unit 231, the detection unit 232detects a gaze of a user based on a captured image input from the gazesensor 11. The detection unit 232 determines whether a direction of thegaze movement different from a direction of searching the guide regionhas been detected, based on the guide region information, and theposition and the direction of the detected gaze point (Step S14). If thedirection of the gaze movement different from the direction of searchingthe guide region has been detected (YES at Step S14), the detection unit232 outputs the guide region information and a highlighting instructionto the highlighting unit 233.

Upon input of the guide region information and the highlightinginstruction from the detection unit 232, the highlighting unit 233generates highlighting information indicating that the guide region inthe guide region information is to be highlighted, and outputs thegenerated highlighting information to the display control unit 231. Uponinput of the highlighting information from the highlighting unit 233,the display control unit 231 highlights the guide region according tothe highlighting information (Step S15). If the direction of the gazemovement different from the direction of searching the guide region hasnot been detected (NO at Step S14), the detection unit 232 continuesdetection of the user's gaze.

The reception unit 234 receives a specific operation for identifying thefirst key data based on the operation information including the penpoint position input from the pen input sensor 12 (Step S16). Uponreception of the specific operation, the reception unit 234 outputsspecific information identifying the first key data to the highlightingunit 233. Upon input of the specific information from the reception unit234, the highlighting unit 233 generates specific display informationfor the identified first key data, and outputs the generated specificdisplay information to the display control unit 231. Upon input of thespecific display information from the highlighting unit 233, the displaycontrol unit 231 highlights the identified first key datacorrespondingly to the specific display information.

Further, upon input of the specific information from the reception unit234, the detection unit 232 changes the first state to a first statecorresponding to the region of the second key data. That is, thedetection unit 232 changes the guide region to the region of the secondkey data. The detection unit 232 outputs a change instruction of theguide region to the display control unit 231. Upon input of the changeinstruction from the detection unit 232, the display control unit 231displays a reference region guide on the second axis of the data list(Step S17).

The detection unit 232 determines whether the direction of the gazemovement different from the direction of searching the guide region hasbeen detected (Step S18). If the direction of the gaze movementdifferent from the direction of searching the guide region has beendetected (YES at Step S18), the detection unit 232 outputs the guideregion information and a highlighting instruction to the highlightingunit 233.

Upon input of the guide region information and the highlightinginstruction from the detection unit 232, the highlighting unit 233generates highlighting information indicating that the guide region inthe guide region information is to be highlighted, and outputs thegenerated highlighting information to the display control unit 231. Uponinput of the highlighting information from the highlighting unit 233,the display control unit 231 highlights the guide region according tothe highlighting information (Step S19). If the direction of the gazemovement different from the direction of searching the guide region hasnot been detected (NO at Step S18), the detection unit 232 continuesdetection of the user's gaze.

The reception unit 234 receives a specific operation for identifying thesecond key data based on the operation information including the penpoint position input from the pen input sensor 12 (Step S20). Uponreception of the specific operation, the reception unit 234 outputsspecific information identifying the second key data to the highlightingunit 233. Upon input of the specific information from the reception unit234, the highlighting unit 233 generates specific display informationfor the identified second key data, and outputs the generated specificdisplay information to the display control unit 231. Upon input of thespecific display information from the highlighting unit 233, the displaycontrol unit 231 highlights a column at a point of intersection of theidentified first key data and second key data (Step S21), to finish theprocessing. Accordingly, the work assistance system 2 can assist thework of identifying the desired data from the data list.

Thus, the work assistance system 2 receives the specific operation withrespect to any one piece of data in the displayed data list. Further,the work assistance system 2 changes the content of the first state,which is a determination criterion at the time of changing the region tobe highlighted, correspondingly to the reception of the specificoperation. As a result, because a guide is displayed in order of itemsof data for assistance, a work of identifying more desired data can beassisted.

Further, the work assistance system 2 prompts the operator to perform anentry operation to the name or address entry column and calculates acorrection value for detecting the gaze position of the individualoperator, based on a relation between the pen point position and thegaze position while the entry operation by the operator to the name oraddress entry column is being performed. Further, the work assistancesystem 2 detects a direction of the gaze movement of the operator withrespect to the displayed data list by using the calculated correctionvalue. As a result, the user's gaze position and the direction can bedetected more accurately.

Further, in the work assistance system 2, the operator's condition canbe presented on other information processing devices and informationpresentation screens on a place where the system has been installed (forexample, stores), in synchronization with the highlighting to a stateviewable by the operator. As a result, for example, a dedicatedassistant can directly assist the user with the work.

In the respective embodiments described above, a two-dimensional listhas been used as a data list to be referred to. However, the data listis not limited thereto. For example, the data list to be referred to canbe a one-dimensional table such as a correspondence table betweenprefecture names and code numbers thereof, or a table ofthree-dimensions or more.

Further, in the respective embodiments described above, the data list tobe referred to is one kind; however, the data list is not limitedthereto. For example, when a plurality of code numbers are to be enteredin a form, a code list corresponding to each of the code numbers can bedisplayed as the data list at the time of entering each of the codenumbers.

Further, respective constituent elements of respective parts illustratedin the drawings do not always need to have physically the sameconfiguration as illustrated in the drawings. That is, the specific modeof distribution and integration of the parts is not limited to theillustrated ones, and all or a part thereof can be configured to befunctionally or physically distributed or integrated in an arbitraryunit, according to various kinds of load and the status of use. Forexample, the detection unit 232 and the reception unit 234 can beintegrated with each other. Further, respective processes illustrated inthe drawings are not always performed in the order described above. Theprocesses can be performed simultaneously or can be performed whilechanging the order, within a range of not causing any contradiction inthe contents of the processes.

Furthermore, all or an arbitrary part of each processing functionexecuted by respective devices can be realized by a CPU (or amicrocomputer such as an MPU and an MCU (Micro Controller Unit)). It isneedless to mention that all or an arbitrary part of each processingfunction can be realized by a program analyzed and executed in the CPU(or a microcomputer such as an MPU and an MCU), or realized as hardwareby a wired logic.

Various processes explained in the embodiments described above can beachieved by executing a program prepared in advance by a computer. Inthe following descriptions, an example of a computer that executes aprogram having functions identical to those in the above embodiments isexplained. FIG. 9 is a diagram illustrating an example of a computerthat executes a computer-readable recording medium.

As illustrated in FIG. 9, a computer 300 includes a CPU 301 thatperforms various types of arithmetic processing, an input device 302that receives data input, and a monitor 303. Further, the computer 300includes a medium reader 304 that reads a program or the like from astorage medium, an interface device 305 for connection to variousdevices, and a communication device 306 for wired or wireless connectionto other information processing devices or the like. The computer 300also includes a RAM 307 for temporarily storing various pieces ofinformation, and a hard disk device 308. The respective devices 301 to308 are connected to a bus 309.

The hard disk device 308 stores therein a computer-readable recordingmedium having the same functions as those of the respective processingunits of the display control unit 131, the detection unit 132, and thehighlighting unit 133 illustrated in FIG. 1. Further, the hard diskdevice 308 can store therein a computer-readable recording medium havingthe same functions as those of the respective processing units of thedisplay control unit 231, the detection unit 232, the highlighting unit233, the reception unit 234, and the calculation unit 235 illustrated inFIG. 6. The hard disk device 308 also stores therein the list storageunit 121 and various pieces of data for realizing a computer-readablerecording medium.

The input device 302 has, for example, the same functions as those ofthe pen input sensor 12 illustrated in FIG. 6 and receives input ofvarious pieces of information such as the operation information from auser of the computer 300. The monitor 303 has, for example, the samefunctions as those of the display unit 10 illustrated in FIG. 1 anddisplays various screens such as the reference screen to the user. Theinterface device 305 is connected to, for example, the gaze sensor 11illustrated in FIG. 1 or FIG. 6. The communication device 306 isconnected to, for example, a network (not illustrated) to transmit andreceive various pieces of information to and from other informationprocessing devices.

The CPU 301 reads out the respective programs stored in the hard diskdevice 308 and executes the programs by developing the programs in theRAM 307, to perform various types of processing. These programs cancause the computer 300 to function as the display control unit 131, thedetection unit 132, and the highlighting unit 133 illustrated in FIG. 1.Further, these programs can cause the computer 300 to function as thedisplay control unit 231, the detection unit 232, the highlighting unit233, the reception unit 234, and the calculation unit 235 illustrated inFIG. 6.

The computer-readable recording medium described above does not alwaysneed to be stored in the hard disk device 308. For example, the computer300 can read a program stored in a storage medium readable by thecomputer 300 and execute the read program. A storage medium readable bythe computer 300 corresponds to a portable recording medium such as aCD-ROM, a DVD disk, or a USB (Universal Serial Bus) memory, asemiconductor memory such as a flash memory, and a hard disk drive, forexample. Further, the computer-readable recording medium can be storedin a device connected to a public circuit, the Internet, a LAN, or thelike and the computer 300 can read the computer-readable recordingmedium therefrom to execute the computer-readable recording medium.

It is possible to assist a work of identifying desired data from a datalist.

All examples and conditional language recited herein are intended forpedagogical purposes of aiding the reader in understanding the inventionand the concepts contributed by the inventor to further the art, and arenot to be construed as limitations to such specifically recited examplesand conditions, nor does the organization of such examples in thespecification relate to a showing of the superiority and inferiority ofthe invention. Although the embodiments of the present invention havebeen described in detail, it should be understood that the variouschanges, substitutions, and alterations could be made hereto withoutdeparting from the spirit and scope of the invention.

What is claimed is:
 1. A work assistance system comprising: a memory;and a processor coupled to the memory, wherein the processor executes aprocess comprising: displaying a data list in which pieces of data arearranged according to a predetermined rule, in a state being viewable byan operator; detecting a direction of a gaze movement of the operatorwith respect to the displayed data list; and changing a region to behighlighted in the data list, depending on whether the detecteddirection of the gaze movement is in a predetermined state.
 2. The workassistance system according to claim 1, wherein, in the changing of aregion to be highlighted, highlighting is not performed when thedirection of the gaze movement is in a first state, and a specificregion is highlighted in the data list when the direction of the gazemovement is in a state different from the first state.
 3. The workassistance system according to claim 2, wherein the first state is astate decided according to the predetermined rule followed by the datalist.
 4. The work assistance system according to claim 2, the processfurther comprising: receiving a specific operation with respect to anyone piece of data in the displayed data list, wherein in the changing ofa region to be highlighted, a content of the first state, which is adetermination criterion at a time of changing a region to behighlighted, is changed in response to reception of the specificoperation.
 5. The work assistance system according to claim 1, whereinthe highlighting is at least one display among a display in whichluminance is increased, a display in which luminance is periodicallychanged, a display in which color is changed, and a display in which athickness of a frame enclosing a region is changed, as compared withother regions in the data list.
 6. The work assistance system accordingto claim 1, the process further comprising: prompting the operator toperform an entry operation to a name or address entry column, andcalculating a correction value for detecting the gaze position of theindividual operator based on a relation between a pen point position andthe gaze position while the entry operation to the entry column is beingperformed by the operator, wherein in the detecting, a direction of thegaze movement of the operator with respect to the displayed data list isdetected by using the calculated correction value.
 7. A work assistancemethod comprising: displaying a data list in which pieces of data arearranged according to a predetermined rule, in a state being viewable byan operator, using a processor; detecting a direction of a gaze movementof the operator with respect to the displayed data list, using theprocessor; and changing a region to be highlighted in the data list,depending on whether the detected direction of the gaze movement is in apredetermined state, using the processor.
 8. The work assistance methodaccording to claim 7, wherein, in the changing of a region to behighlighted, highlighting is not performed when the direction of thegaze movement is in a first state, and a specific region is highlightedin the data list when the direction of the gaze movement is in a statedifferent from the first state.
 9. The work assistance method accordingto claim 8, wherein the first state is a state decided according to thepredetermined rule followed by the data list.
 10. The work assistancemethod according to claim 8, further comprising: receiving a specificoperation with respect to any one piece of data in the displayed datalist, using the processor, wherein in the changing of a region to behighlighted, a content of the first state, which is a determinationcriterion at a time of changing a region to be highlighted, is changedin response to reception of the specific operation.
 11. The workassistance method according to claim 7, wherein the highlighting is atleast one display among a display in which luminance is increased, adisplay in which luminance is periodically changed, a display in whichcolor is changed, and a display in which a thickness of a frameenclosing a region is changed, as compared with other regions in thedata list.
 12. The work assistance method according to claim 7, furthercomprising: prompting the operator to perform an entry operation to aname or address entry column, and calculating a correction value fordetecting the gaze position of the individual operator based on arelation between a pen point position and the gaze position while theentry operation to the entry column is being performed by the operator,using the processor, wherein in the detecting, a direction of the gazemovement of the operator with respect to the displayed data list isdetected by using the calculated correction value.
 13. A non-transitorycomputer-readable recording medium having stored therein a program thatcauses a computer to execute: displaying a data list in which pieces ofdata are arranged according to a predetermined rule, in a state beingviewable by an operator; detecting a direction of a gaze movement of theoperator with respect to the displayed data list; and changing a regionto be highlighted in the data list, depending on whether the detecteddirection of the gaze movement is in a predetermined state.
 14. Thenon-transitory computer-readable recording medium according to claim 13,wherein, in the changing of a region to be highlighted, highlighting isnot performed when the direction of the gaze movement is in a firststate, and a specific region is highlighted in the data list when thedirection of the gaze movement is in a state different from the firststate.
 15. The non-transitory computer-readable recording mediumaccording to claim 14, wherein the first state is a state decidedaccording to the predetermined rule followed by the data list.
 16. Thenon-transitory computer-readable recording medium according to claim 14,wherein the program causes the computer to further execute: receiving aspecific operation with respect to any one piece of data in thedisplayed data list, wherein in the changing of a region to behighlighted, a content of the first state, which is a determinationcriterion at a time of changing a region to be highlighted, is changedin response to reception of the specific operation.
 17. Thenon-transitory computer-readable recording medium according to claim 13,wherein the highlighting is at least one display among a display inwhich luminance is increased, a display in which luminance isperiodically changed, a display in which color is changed, and a displayin which a thickness of a frame enclosing a region is changed, ascompared with other regions in the data list.
 18. The non-transitorycomputer-readable recording medium according to claim 13, wherein theprogram causes the computer to further execute: prompting the operatorto perform an entry operation to a name or address entry column, andcalculating a correction value for detecting the gaze position of theindividual operator based on a relation between a pen point position andthe gaze position while the entry operation to the entry column is beingperformed by the operator, wherein in the detecting, a direction of thegaze movement of the operator with respect to the displayed data list isdetected by using the calculated correction value.